AUTONOMOUS DIAGNOSTIC IMAGING PERFORMED BY UNTRAINED OPERATOR USING 
AUGMENTED REALITY AS A FORM OF “JUST-IN-TIME” TRAINING 


BACKGROUND 


Currently, astronauts receive remote guidance, two-way 
communication with experts in Mission Control in near-real time to 
perform medically-required or research-directed procedures. Remote 
guidance has been used successfully to direct International Space 
Station crewmembers to obtain ultrasound images of various 
anatomical structures and optical coherence tomography (OCT) 
images of the eye to investigate space flight-induced adaptations 
associated with Human Research Program Risks and Knowledge 
Gaps. However, because of the increased distance between Mission 
Control and the spacecraft during exploration missions beyond low 
Earth orbit, communication delays may be as great as 20 minutes. 
The lack of real-time communication will prevent direct assistance 
from terrestrial experts to complete these tasks, thus requiring 
astronauts to perform tasks more autonomously. With the amount 
and complexity of pre-flight training required of astronauts for 
Spacecraft operations and mission objectives, adequate training prior 
to launch covering every potential diagnostic scenario is not realistic. 
Some form of Just-in-Time training (JIT) is needed through which 
astronauts can access a library of tutorials to guide them through 
technically-challenging imaging procedures in which they may have 
little or no previous training. 


PURPOSE 


To facilitate the need for autonomous operation, astronauts will 
need to employ aspects of JIT training. The delivery of instructional 
material and increased situational awareness and represents a 
significant improvement in the ability of astronauts to conduct 
medical imaging autonomously. 


INTRODUCTION 


We are addressing the Human Factors and Performance Team 
“Risk of performance errors due to training deficiencies” by creating 
augmented reality (AR) guidance materials for ultrasound and 
(OCT) imaging. 


Features of the AR tutorial include: 

¢ A detailed audio and video tutorial that is viewed through a 
holographic heads-up display while performing the imaging 
procedure as opposed to reviewing the material beforehand 
(Figure 1). 
Hands-free guidance will include step-by-step procedures 
describing equipment setup and operation, subject positioning, 
and image acquisition. 
Procedures are accompanied by verbal guidance of scan 
technique and protocol, reference images or video, and 
photographs of equipment alignment. 
Examples of common imaging errors and pitfalls are included 
along with instruction to achieve a better image. 
The AR system highlights controls on the actual ultrasound 
keyboard or OCT system when needed (Figure 3) 
The AR system highlights starting ultrasound scanning locations 
on the subject’s body to improve the scanner’s ability to find an 
adequate image. 
The AR system will provide real-time feedback to the scanner 
based on the image generated on the OCT system. 
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Figure 1. Depiction of how instructional material is seem by the user on the heads-up 
display of the AR headset, while still being able to view the subject and the ultrasound 
device. 


SPECIFIC AIMS 


Specific Aim 1: Develop an audio-visual tutorial using augmented 
reality glasses that guides non-experts through an abdominal trauma 
ultrasound protocol. 


specific Aim 2: Develop an audio-visual tutorial using augmented- 
reality glasses to guide an untrained operator through the setup and 
acquisition of an OCT protocol. 


Specific Aim 3: Compare the quality of abdominal ultrasound and 
OCT images acquired by untrained operators using augmented 
reality guidance during ground-based testing to images acquired 
using current JIT techniques. 
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Figure 2. An AR headset being used to perform ultrasound imaging. 


Figure 3. Ultrasound system as viewed through AR headset during operation.. 


METHODS 


An AR audio-visual tutorial and laptop-based JIT tutorial will be 
developed to guide untrained operators through a trauma 
ultrasound protocol and through acquisition of OCT images. 

One group of 25 untrained subjects without previous experience 
in ultrasound or OCT will use traditional laptop-based JIT training 
to perform ultrasound and OCT imaging. 

A second group of 25 untrained subjects without previous 
experience in ultrasound or OCT will use AR guidance to perform 
ultrasound and OCT imaging. 

Image quality from both groups will be evaluated by experts 
blinded to training type used for image acquisition. 

The time required to acquire quality images using AR guidance 
and traditional JIT training will be compared to determine whether 
one modality is more efficient. 


Figure 4. An OCT exam being performed under AR guidance. 


EXPECTED OUTCOME 


AR-guided procedures will generate higher quality images as 
compared to traditional JIT training. 
AR-guided procedures will be completed in less time. 
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